Collateral Data Combined with User Characteristics to Select Web Site 
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Field of the invention: 

The present invention relates to electronic systems and more particularly to the 
use of collateral data and a data base of personal characteristics to provide 
relevant information to a user. 



Background of the Invention; 

Various techniques are known for embedding collateral information in images, in 
files of text, in audio information and in other types of digital information. Digital 
watermarking (which are a particular type of steganographic data encoding) are 
one way of imbedding collateral information in digital files. There are many types 
of digital watermarks. For example, prior application 09/127,502, filed July 31, 
1998, now published as WO0007356, (which is assigned to the assignee of the 
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present invention) shows a technique by which very fine lines can be printed on a 
medium to slightly change the medium's apparent tint, while also conveying 
digital data. Co-pending application 09/074,034, filed May 6, 1998, now 
published as WO 9953428, details how the contours of printed imagery can be 
adjusted to convey digital data. Issued patent 5,850,481 details how the surface 
of paper or other media can be textured to convey optically-detectable binary 
data. Issued patents 5,862,260, 5,841 ,886 and 5,809,160 detail various 
techniques for steganographically encoding photographs and other imagery. 
There are a great man other patents and a large body of literature which describe 
various other watermarking and steganographic techniques. 

The technology and science for collecting user preferences using the internet 
and other data gathering mechanisms is also well developed. For example US 
patents 5,918,014 and 5,933,811 and PCT publication WO 00/38074 describe 
systems that collect data concerning users and which then provide to the users 
advertisements directed to what the system decides is their particular interests 
and preferences. As used herein the term "user characteristics" means any 
information about a user's characteristics or preferences such as the type of 
information discussed in the article "Privacy 2000 In Web We Trust" published in 
the magazine "PC World", June 2000, pages 103 to 108. 

Systems such as the system described in co-pending application 09/571 ,422 
filed 05/15/00, utilize collateral data stored in printed images to carry a URL 
(Universal Resource Locator) address (or a dynamic index to a database entry 
storing such addresses). When a user of the system directs a camera at an 
image containing the collateral data, the user's web browser is directed to a 
particular web site. 
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Summary of the present Invention: 

The present invention provides a system where collateral information (in a file or 
image) is used together with information concerning a particular user's 
characteristics to direct a user to a particular web site. The site to which a user is 
directed is selected based both upon the original collateral information and based 
upon information from the data base of user characteristics. In one embodiment, 
a file or data stream includes collateral data that identifies the URL of a central 
web site. When a file containing the collateral data is transferred to a browser, 
the central web site is accessed and it interrogates a data base containing 
various information including user preferences. The output of the data base 
search is another URL that directs the browser that received the original 
collateral data to another web site which then sends information to the user. In 
an alternate embodiment, a user uses a hand held digital camera to gather 
information from images such as magazine covers or product boxes. The 
images contain digital watermarks that carry collateral data. The collateral data 
is used to interrogate either a local or a remote data base which contains 
information pertinent to the particular user's characteristics. The output from the 
data base is then presented to the user. Thus, the collateral data from the 
original images is transformed into information particularly pertinent to the 
particular user. 

Brief Description of the Figures 

Figure 1 is an overall system diagram of a first embodiment of the invention. 
Figure 2 is a flow diagram of the operation of the system. 
Figure 3 is a system diagram of a second embodiment. 
Figure 4 illustrates another embodiment of the invention. 

Detailed Description: 

An overall system diagram of a first embodiment is shown in Figure 1 . The 
system includes a conventional Internet web site 101 and an internet browser 
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program 102 operated by a user (who is not shown in the Figure). It should be 
understood that when implemented on the Internet, the system would generally 
include many such web sites and many such browsers. The system also 
includes a central server 104 which includes data bases 104A and 104B. Finally 
the diagram shows a web site 105 which includes information of particular 
interest to the user of browser 102. Note, in an internet implemented system 
there would be a relatively large number of web sites 105, each having 
information for particular users that have particular sets on interest and 
characteristics. 

The browser 102 includes a watermark reading plug in program. If a web page 
or a file is sent to browser 102 which contains a watermark, the watermark is 
detected and read by the watermark reading program in browser 102. Co- 
pending application 09/571,422 filed 5/15/2000 which is incorporated herein by 
reference describes such a program. 

The flow diagram in Figure 2 illustrates the operation of the system. The process 
starts when browser 102 requests a file from web site 101 . The file can be a text 
file, an audio file or an image file. The file stored on web site 101 includes 
collateral data. This data may be in the form of a watermark in an image or audio 
file or as meta data in any type of file. The simplest situation is if the requested 
web page contains a watermarked image. It should however be noted that the 
invention can work with other types of collateral data transferred from server 101 
to browser 102, such as for example, when audio data is transferred. 

As indicated by blocks 202 and 203, the file is sent to the browser and the 
browser extracts the collateral data from the file. If the data is in the form of a 
watermark in an image file or in audio data, the browser would utilize a plug-in 
which can read the watermark and extract the collateral data. The details of the 
watermark reading program per se are known in the art and are not part of the 
present invention. 
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The collateral data extracted by the watermark reading program includes the 
URL of central server 104 and an identifier of the file or image that contained the 
watermark. First data base 104A is interrogated to find information concerning 
the organization that registered the particular watermark. This is similar to the 
process described in co-pending application 09/571,422 filed 5/15/00. 

Central server 104 also includes a second data base 104B which is indexed 
according to Globally Unique Identifiers (GUIDs). These identifiers allow the 
system to track references that come from a particular browser 102 without 
obtaining or using the actual name or e-mail address of the user. Such Globally 
Unique Identifiers (GUIDs) are in widespread use by firms that provide 
advertisements on the World Wide Web. For example see an article in the 
magazine PCWorld June 2000, page 103 to 108 entitled "Privacy 2000 in Web 
We Trust". The data base 104B includes information relevant to each GUID. The 
information in data base 104B is acquired in a conventional manner. 

The data base 104A and 104B also includes a list or data base of alternate sites 
105. The data bases 104A and 104B include for each site listed, the 
characteristics of the users that the site owner would like to reach and the 
identifiers from watermarks. Thus when the collateral data in a particular image 
causer browser 102 to contact central server 104, the central server 104 makes 
two matches. First the identifier in the watermark is matched to registered 
identifiers to identify which particular image or web page contained the 
watermark. Second, from the GUID the server can determine the characteristics 
of the user. These two sets of information are used to determine the alternative 
site to which the user is directed. 

The following is a very simple example which illustrates the operation of the 
system. Assume that the GUIDs identify only three characteristics of users that 
are designated "characteristic one", "characteristic two" and "characteristic three". 
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Also assume that the watermarks read by browser 102 have only three different 
identifiers designated Identifier A, Identifier B and Identifier C. The alternative 
site 105 to which the user would be directed would be determined by server 104 
from the information in data bases 104A and 104B as shown by the following 
table. 





Characteristic 
one 


Characteristic 
Two 


Characteristic 
three 


Identifier A 


Site S1 


Site S4 


Site S7 


Identifier B 


Site S2 


Site S5 


Site S8 


Identifier C 


Site S3 


Site S6 


Site S9 



Table 1 



The point is that server 104 has available two sets of information. The first 
identifies the image or file from which the collateral data was obtained and the 
second is the characteristics of the user. These two bits of information can be 
combined as shown above to select an appropriate site (above identified as sites 
S1 to S9) to which the user should be directed. 

It is specifically noted that in any practical situation there will be many more 
Identifiers and many more characteristics or more probably combinations of 
characteristics. Furthermore, many different combinations might point to the 
same web site. As shown above each different combination pints to a different 
web site S1 to S9. 

Finally as indicated by block 206 in Figure 2, data from the selected alternate 
web site 105 is sent to the browser 102. The net result of the process is that the 
user who requests a web page or file from web site 101 will also receive certain 
collateral data which may be in the form of a watermark in the web page or file. 
This collateral data results in a request to server 104 which using the GUID of 
browser 102 and information from the watermark, locates an appropriate 
alternate web site 105. Information from the selected alternate web site 105 is 
then sent to browser 102. 
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An alternate embodiment of the invention is shown in Figure 3. This alternate 
embodiment of the invention utilizes some of the components from the Media 
Bridge technology commercially available from Digimarc Corporation , Tualatin 
Oregon. With the Media Bridge technology, one can capture a digital image and 
read a digital watermark which is included in the image. The digital image can be 
captured with a commercially available PC camera. 

In the embodiment shown in Figure 3, a PC camera 301A located in a hand held 
device 301 captures images, which can, for example, be the front page of 
magazines 302A to 302X. The hand held device includes a watermark reading 
program 301 B and a data base 301 C that stores data concerning the user's 
personal preferences. The hand held device 301 also includes a wireless 
Internet connection which can connect to a web sites 303. 

Each of the magazine front page images 302A to 302X includes a digital 
watermark which specifies a particular URL. The camera 301 A reads the 
watermarks in the images, 302A to 302X, obtains the information from the 
watermark and contacts one of the web sites 303A to 303X. The particular web 
sites contacted depends upon the information read from the watermarks. Each 
web site includes an expanded index of the information in the associated 
magazine. This index is sent to handheld device 301. The hand held device 301 
compares the information in the expanded magazine indexes to the information 
in the data base 301 C and informs the user which particular magazine has 
articles which match the personal preferences stored in data base 301 C. 

Thus a user can scan a rack of magazine covers with device 301 . Device 301 
reads the URL specified by the watermark on each magazine cover and then 
obtains an index of the particular magazines from a remote data base. The index 
is compared to the users personal preferences stored in hand held device 301 
and the user is informed as to which magazine matches the stored preferences. 
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It is noted that the designations 302A to 302X and 303A to 303X is used to 
illustrate that the number of magazines and the number of associated web sites 
is an arbitrary number suited to a particular application. The designations are 
meant to illustrate that there is one web site 303 (or one web page) for each 
magazine cover 302. 

It is noted that as new issues of the various magazines appear, the information in 
web sites 303A to 303X change. Each magaize therefore has an identifier which 
directs the system to a particular web site. The web site will have the current 
information for that magazine. 

The present embodiment goes to a different web site for each magazine index. It 
noted that the various magazine indexes could be stored as different web pages 
on a single web site. 

In an alternate embodiment, the watermarks on the magazine images includes 
coded information about the content of the magazines. The hand held device 
can then compare the magazine content to the stored preferences and indicate a 
match without accessing a web site. In another alternate embodiment, the 
content can be stored in the hand held device and periodically synchronized with 
a remote source. 

It should be specifically noted that while the foregoing specification focuses on 
applications employing digital watermarking, the present invention can 
alternatively employ other data encoding techniques, including 1D and 2D 
barcodes, magnetic ink character recognition (MICR), optical character 
recognition (OCR), optical mark recognition (OMR), radio frequency identification 
(RF/ID), UV/IR identification technologies, data glyphs, organic transistors, 
magnetic stripe, etc., depending on the particular application requirements. 
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. Uilf: 



Co-Pending application 09/571,422 filed 05/15/00 describes a system that reads 
collateral data and which has a router and registration data base to determine an 
appropriate URL when a particular object is viewed. The content of application 
09/571,422 filed 05/15/00 is hereby incorporated herein by reference. The 
present invention can be applied as an extension of the system shown in 
application 09/571 ,422 filed 05/1 5/00. 

Figure 4 illustrates another alternate embodiment of the present invention. With 
the system shown in Figure 4, there is equipment 401 at a remote location 
connected to equipment 402 which is at a central location. A user operates an 
originating device 412 at the remote location. The originating device 412 
includes a PC camera 41 2A which can acquire an electronic image of a printed 
advertisement 429. The originating device 412 includes a watermark reading 
program 412B and an internet browser 412C. 

The collateral data read from the electronic image generated by camera 41 2A is 
used to generate a URL which directs the browser 412C to server 414 via the 
internet 432. When server 414 receives a request from originating device 412, it 
uses the GUID information to interrogate a data base 418 which contains 
information about users. The server 414 also interrogates a registration data 
base 417 which contains information relevant to the particular collateral data read 
from the object 429. These two sets of information are combined as illustrated 
by the simple example in table 1 above. 

The sever can therefore respond to a request from originating device 412 by 
directing the browser 41 2C to a web site (not shown in Figure 4) which is 
determined by both the collateral data in the object 429 and the user's 
characteristics and preferences in data base 418. 

The content of data base 418 can be generated in any of the ways known in the 
art for obtaining information about particular users. It should be noted that as 
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used herein the term "user characteristics" means any information about a user's 
characteristics, preferences, interests, patterns or habits. Furthermore, the term 
"user" in general means the person that operates or utilizes a particular terminal 
or system. 

It is noted that in some embodiments, the system has two data bases, one of 
which has information relative to the user and one of which has information 
concerning the detected collateral data. While such embodiments have two data 
bases, it should be understood that these two data bases can be implemented as 
and considered to be a single data base. Thus, as used herein the term data 
base can be understood to mean a single data base or combination of multiple 
databases. 

It should also be noted that the invention may be extended to other forms of 
media data such as audio and video data. For example, as a user listens to 
music or watches a video, digital watermarks imperceptibly embedded in the 
audio or video carry collateral data used to interrogate data bases and retrieve 
information pertinent to the particular listener or viewer. 

Alternative implementations of the invention use fingerprints of the content, such 
as a hash of perceptually relevant features of the content, to derive a content 
identifier from which information particular to the user can be fetched and 
returned. While digital watermarks embed auxiliary data imperceptibly in the 
content by subtly modifying it, fingerprints are dynamically derived from the 
content and do not require embedding of auxiliary data. Both digital watermarks 
and fingerprints can be used in combination to look up related information and to 
find information particular to the user. 

While the invention has been shown and described with respect to several 
different embodiments, it is noted that many other changes in form and detail can 
be made without departing from the sprit and scope to the invention. 
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